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SYSTEM AND METHOD FOR DIGITAL IMAGE TONE MAPPING USING AN 
ADAPTIVE SIGMOID AL FUNCTION BASED ON PERCEPTUAL PREFERENCE 
GUDDELINES 

BACKGROUND OF THE INVENTION 

Field of the Invention 
[0001J The present invention relates generally to digital image tone mapping 

algorithms, and specifically to digital image tone mapping algorithms based on perceptual 
5 preference guidelines. 

Description of Related Art 
[0002] Digital image tone mapping refers to the process of mapping captured scene 

luminance levels to the luminance or density levels of an output device (display or printer). 
10 Digital image tone mapping is necessary due to the fact that scene luminance ranges very 
seldom match the luminance range of the output device. Digital image tone mapping 
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algorithms can be implemented within any digital device to produce visually pleasing output 
images. For example, such devices can include digital cameras, scanners, digital camcorders, 
printers capable of printing digital images and digital televisions. In addition, digital tone 
mapping algorithms are frequently used in image enhancement software applications. 
5 [0003] Since the human eye adapts to different light levels in different ways, 

perceptual factors must be taken into account when attempting to generate a rendered image 
that looks "right" with respect to the original scene. Depending on the application, image tone 
mapping typically has one of three goals: (1) appearance match; (2) subjective preference; or 
(3) information preservation. The goal of appearance matching strives to make the rendered 

10 image as perceptually similar as possible to the original scene. This is usually an implicit goal 
in consumer imaging and image synthesis applications. Taking into account subjective 
preferences allows the image to look as pleasing as possible to the viewer. This is usually 
desirable in consumer imaging and commercial photography. If the goal is information 
preservation, the algorithm seeks to preserve or enhance the details at all regions and all 

1 5 luminance levels of an image. This is most often requested in medical imaging, satellite 
imaging, and archiving. 

[0004] Many existing tone mapping algorithms focus on achieving an appearance 

match between the original image and the rendered image. There are at least two perceptual 
factors typically considered in such algorithms: (1) global luminance adaptation; and (2) local 
20 luminance adaptation. The overall global luminance level of the scene influences the 

adaptation state of the eye. Two aspects of such global luminance adaptation have significant 
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impact on tone mapping, brightness and spatial contrast. 

[0005] First, the brightness function is different at different scene luminance levels. 

Perceived brightness, which corresponds to the viewer's perceived strength of light, is roughly 
a power function of physical luminance (Steven's law). The exponent for such a power 
5 function is larger when the overall luminance level is higher. When rendering an image on a 
lower luminance device, the exponent must be adjusted to accommodate such differences. In 
addition to the brightness function change, the spatial contrast sensitivity of the eye also 
changes as it adapts to different ambient luminance levels. When the ambience is bright, the 
eye perceives the high spatial frequency components (details) of an image better than when the 
10 ambience is dark, i.e., the visual acuity of the eye improves with better ambient lighting. Also, 
the contrast threshold, i.e., the minimum contrast needed to detect components of the image, 
decreases with increased luminance level. To render a bright image onto a lower luminance 
device, the luminance contrast of the details in the image can be enhanced to account for these 
effects. 

1 5 [0006] Two different tone mapping algorithms developed by Jack Holm and Tumblin 

& Rushmeier, respectively, account for the brightness function change by adjusting the 
curvature of the tone curves based on the absolute luminance level of the scene. Each of these 
tone mapping methods is described separately in Holm, J., "Photographic Tone and Colour 
Reproduction Goals," CIE Expert Symposium on Colour Standards for Image Technology, 

20 pp. 51-56 (1996); and Tumblin, J. and Rushmeier, H., "Tone Reproduction for Realistic 

Images," IEEE Computer Graphics and Applications, 13(6):42-48 (1993), both of which are 
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hereby incorporated by reference. These algorithms have the benefit of creating the proper 
overall sensation of brightness or darkness corresponding to the original image, which is 
desirable in high-end digital imaging. However, both require accurate information about the 
absolute luminance level of the original image. In a digital camera, it is possible to estimate 
5 absolute luminance levels of image pixels from the raw pixel values and the camera's capture 
settings such as aperture, exposure time, lens properties, etc. However, in low cost cameras, 
such calculations are often not available due to the added complexity and cost. Therefore, a 
global luminance adaptation solution that focuses on preference is more practical. 
[0007] In tone mapping algorithms based on preference, the goal is to achieve a certain 

10 set of image properties liked by viewers. The widely used histogram equalization method can 
be categorized as such an algorithm. The histogram method is based on the observation that 
most "good" images have a luminance histogram that fully occupies the output dynamic range. 
The algorithm adjusts image gray levels to move the histogram shape toward a flat, Gaussian, 
or some other predetermined shape. Of course, how well such a method works depends on 

1 5 whether the assumption is true that every "good" image has the same histogram. The method 
does well on images that have a symmetric and well-distributed histogram, but makes images 
look unnatural when there are large areas of dark or light background in the image, which bias 
the histogram toward one side. 

[0008] A modified histogram equalization method developed by Larson, et al. is more 

20 robust than traditional histogram equalization methods. Larson's method limits the amount of 
gray level adjustments allowed in the tone mapping. The amount of gray level adjustments are 
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limited based on luminance contrast sensitivity measurements. In addition, one variation of 
this method also accounts for the change in visual acuity under different illumination levels. 
Reference is made to Larson, G. W., Rushmeier, H. and Piatko, C , "A Visibility Matching 
Tone Reproduction Operator for High Dynamic Range Scenes," IEEE Transactions on 
5 Visualization and Computer Graphics, 3(4):291-306 (1997), which is incorporated by 

reference. However, the computation is iterative, and thus the implementation is costly and 
slow. In addition, the modified histogram equalization method also requires accurate absolute 
luminance level information. Thus, although this modified histogram equalization method 
creates a more accurate appearance match, it does so at the cost of higher computational 
1 0 complexity. 

[0009] The local luminance adaptation perceptual factor considers the fact that the eye 

looks at an image by scanning around. The eye can rapidly adapt to the luminance level of 
small regions in the original scene to enable regions in the shadows and in the highlights to be 
clearly visible to the eye. In the rendered image, both the dynamic range and the adaptation 

1 5 environment are different. Therefore, to fully imitate the eye's adaptation process, the 
luminance levels of an image are adjusted according to its local luminance levels. 
[0010] Various local tone mapping algorithms, such as Tumblin's detail-preserving 

contrast reduction method (Tumblin, J. and Turk, G., "LOS: A Boundary Hierarchy for 
Detail-Preserving Contrast Reduction," Computer Graphics Proceedings, SIGGRAPH 99, pp. 

20 83-90, Los Angeles, CA, USA (1999), which is incorporated by reference), and various 
algorithms based on the retinex theory have attempted to imitate the local luminance 
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adaptation process. Reference is made to Jobson, D., Rahman, Z. and Woodell, G., "A 
Multiscale Retinex for Bridging the Gap Between Color Images and Human Observation of 
Scenes," IEEE Transactions on Image Processing, 6(7):965-976 (1997); and Rahman, Z , 
Jobson, D. and Woodell, G, "Multi-Scale Retinex for Color Image Enhancement, 
5 Proceedings," International Conference on Image Processing, volume 3, pp. 1003-1006, 
Lausanne, Switzerland (1996) for a discussion of the retinex theory, both of which are 
incorporated by reference. Although these algorithms do preserve the local contrast of 
images, they are iterative methods that also involve the decomposition of different spatial 
resolution components of an image, which is computationally costly. 

10 

SUMMARY OF THE INVENTION 

[0011] A system and method for implementing an adaptive digital image tone mapping 

algorithm that is based on perceptual preference guidelines is provided. The tone curve is 
generated as a sigmoidal function having sigmoidal function parameters (slope and shift) that 

1 5 are determined by original image statistics. Since the parameters are based on original image 
statistics, the algorithm is adaptive. Tone curves generated for different images each have a 
smooth sigmoidal shape, so that the tone mapping process does not change the image 
histogram shape drastically. Therefore, the algorithm is both robust and conservative (e.g., 
while improving the appearance for most images, the algorithm does not make any images 

20 look worse). The sigmoidal function has the form: 
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V ; l + exp(-a(x/100-y?)) ' 
where a is the slope parameter and ft is the shift parameter. The input value x in the sigmoidal 
function varies in the range [0, 100], because the tone curve is generated on an L* scale, 
which has values from 0 to 100. The sigmoidal tone curve calculation can be implemented 
5 efficiently using simple arithmetic operations by pre-calculating and storing various factors 
used in the calculation of a and and by pre-generating a pair of fixed tone curves with two 
extreme slopes and interpolating between the curves. 

BRIEF DESCRIPTION OF THE DRAWINGS 
10 [0012] The disclosed invention will be described with reference to the accompanying 

drawings, which show important sample embodiments of the invention and which are 
incorporated in the specification hereof by reference, wherein: 

[0013] FIGURE 1 is a block diagram illustrating a system utilizing a preferred 

embodiment of the tone mapping algorithm of the present invention; 
1 5 [0014] FIGURE 2 is a flow chart illustrating the steps involved in the tone mapping 

algorithm shown in FIGURE 1 of the drawings; 

[0015] FIGURES 3 A-3C are block diagrams illustrating alternative embodiments for 

implementing the image statistics processor shown in FIGURE 1 of the drawings; 
[0016] FIGURES 4A-4C are flow charts illustrating the steps involved in each of the 

20 embodiments shown in FIGURES 3 A-3C, respectively; 
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[0017] FIGURE 5 is a flow chart illustrating the steps for calculating the slope 

parameter of the tone mapping algorithm of FIGURE 1; 

[0018] FIGURES 6 A and 6B are flow charts illustrating alternative embodiments for 

calculating the shift parameter of the tone mapping algorithm of FIGURE 1; 
5 [0019] FIGURES 7A and 7B are block diagrams illustrating alternative embodiments 

for generating the tone curve of the tone mapping algorithm of the present invention; 
[0020] FIGURES 8A and 8B are flow charts illustrating the steps involved in the 

alternative embodiments shown in FIGURES 7A and 7B, respectively; 
[0021] FIGURE 9 is a graphical representation of a sample tone curve of the type 

1 0 generated by the tone mapping algorithm of the present invention; 

[0022] FIGURE 10 is a graphical representation of two pre-generated tone curves 

plotted on£* axes, in accordance with the embodiment shown in FIGURE 7B and 8B; 
[0023] FIGURE 1 1 A is a graphical representation of output L * levels plotted as a 

function of the slope parameter, in accordance with the embodiment shown in FIGURE 7B 

15 and 8B; and 

[0024] FIGURE 1 IB is a graphical representation of gamma corrected output levels 

plotted as a function of the slope parameter, in accordance with the embodiment shown in 
FIGURE 7B and 8B. 
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DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED 
EXEMPLARY EMBODIMENTS OF THE PRESENT INVENTION 

[0025] The numerous innovative teachings of the present application will be described 

5 with particular reference to the presently preferred exemplary embodiments. However, it 

should be understood that these embodiments provide only a few examples of the many 

advantageous uses of the innovative teachings herein. In general, statements made in the 

specification of the present application do not necessarily delimit any of the various claimed 

inventions. Moreover, some statements may apply to some inventive features, but not to 

10 others. 

[0026] With reference now to FIGURE 1, a digital image tone mapping algorithm 100 

in accordance with preferred embodiments of the present invention is shown implemented 
within a digital image system 10. The digital image system 10 can be any digital device, 
including, but not limited to, a digital camera, scanner, digital camcorder, printer capable of 

1 5 printing digital images or digital television. The algorithm 100 can be included within an 
image processing chip of the digital device, such as a low-end CMOS sensor chip, a CCD 
sensor chip or a separate image processing chip packaged together with the sensor chip, or 
within a software package (driver) associated with the digital device. Alternatively, the digital 
image system 10 can be a computer system having an image enhancement software package 

20 installed thereon in order to enhance tonal characteristics of digital images. 

[0027] The tone mapping algorithm 100 is simple and fast in terms of memory and 

processing time, and uses a global tone mapping method, with one tone curve for all color 
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channels. It should be noted that the tone mapping algorithm 100 can be applied to both color 
images and monochrome images. The tone mapping algorithm 100 produces a smooth and 
consistent tone curve, with gentle curvature, which avoids the extreme contrast changes 
evident in some histogram equalization methods. In addition, the tone mapping algorithm 100 
5 is ideal for video applications since the tone curve remains relatively stable from frame to 
frame by using an adaptive tone which is always sigmoidal in shape. Furthermore, the tone 
mapping algorithm 100 generates images that look pleasing to the user (in terms of brightness, 
contrast and color constancy) without requiring an estimation of the absolute luminance levels 
of the original image. 

1 0 [0028] The tone mapping algorithm 1 00 is based on perceptual preference guidelines, 

such as the preservation of general histogram shape at the middle range by using a centering 
function on i* nth and (100-#)th percentile values, an I* standard deviation of around 20 
and a mean L * value of around 50. It should be understood that the L * scale is simply a 
transformation from a linear scale based on received linear image data. It should be noted that 

15 other perceptual preference guidelines can be used based on the requirements of the user or 
the specific digital image system 10 in which the tone mapping algorithm is implemented. In 
order to produce a tone mapping algorithm 100 that takes into consideration perceptual 
preferences and that complies with computation complexity constraints, a sigmoidal function 
tone mapping method is used, which will be described in more detail hereinbelow with respect 

20 to FIGURES 2-8. 

[0029] With reference now to FIGURE 2, which will be described in connection with 
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FIGURE 1, there is illustrated the steps for mapping received digital image data representing 
the luminance (pixel) values of a digital image to the luminance range of an output device 50, 
using the tone mapping algorithm 100 of the present invention. A digital image source 20, 
such as a digital image capture device or a digital image file, provides digital image data 25 
5 associated with a digital image to a linear transformation device 30 (step 200), which 

transforms the digital image data 25 into linear image data 130 on a linear scale (step 210). It 
should be noted that the original digital image data 25 may be on any scale. Alternatively, the 
digital image data 25 may already be on a linear scale, which obviates the need for the linear 
transformation device 30. This linear image data 130 is later used by a tone mapping operator 
10 40 to map the linear image data 130 onto the luminance range of the output device 50 (step 
250). 

[0030] Once transformed, the linear image data 130 is passed to an image statistics 

processor 120, which determines one or more image statistics 149 based on the original linear 
image data 130 (step 220). These image statistics 149 are used by tone curve parameter 

15 calculation logic 140, along with one or more predefined perceptual preferences in the 
calculation of two tone curve parameters 150, namely, the slope parameter and the shift 
parameter (step 230). The pre-defined perceptual preferences may be defined by an operator 
(e.g., a user or manufacturer of the digital image system 10). The shift parameter shifts the 
tone curve so that under or over-exposed images are brought closer to the right range. The 

20 slope parameter stretches or compresses the tone curve to achieve a desired histogram spread. 
These two tone curve parameters 150 are used by a tone curve generator 160 in generating a 
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tone curve 170 for the image (step 240). The tone curve generator 160 produces a sigmoidal 
tone curve 170 using the following sigmoidal function: 

t(x) = — , (Equation 1) 

l + exp(-a(x/100-/?)) 

5 where a is the slope parameter and /? is the shift parameter. The input value x in the sigmoidal 
function varies in the range [0, 100], because the tone curve 170 is generated on an L* scale, 
which is a measure of the brightness level of the image pixels and has values from 0 to 100. 
Based on the CIE standard formula, the L* scale is approximately a cube root transformation 

of the linear image data: 

10 L * = 1 1 6(~) 1-16, (Equation 2) 

where 7 is linear luminance, and Y„ is the luminance level of the white point. Due to the fact 
that 7 values are normalized by Y„ values for the calculation of L * , the absolute scale of 7 and 
Y„ is not important, as long as both are proportional to the absolute luminance. In 
15 implementation, the raw or color-corrected linear RGB values may take the place of the 7 
values. 

[0031] The L * scale is the preferred scale in the generation of the tone curve 170 for 

several reasons. First, the L * scale is perceptually more uniform than a linear scale. Second, 
most natural images tend to have more symmetric histograms on an L* scale than on a linear 
20 scale, which makes the adjustment of the histogram spread easier (and perceptually more 
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robust) on the L * scale. However, it should be noted that a linear scale may be used by the 
tone curve generator 160 instead of the L * scale described herein. Also, it should be 
understood that the sigmoidal tone curve 1 70 asymptotes to 0 and 1 00 at negative and 
positive infinity. Therefore, once the tone curve 170 is generated, the curve 170 must be 
5 scaled to the [0, 100] range after to make sure 0 maps to 0 and 100 maps to 100 as follows: 



[0032] The sigmoidal shaped tone curve 170 of the type shown in FIGURE 9 that is 

produced by the tone curve generator 160 has several properties that satisfy the constraints 

10 and perceptual preferences discussed above. The sigmoidal shaped tone curve 170 is mostly 
linear at the mid range, with saturation at the dark and bright end if the slope is high. A 
mostly linear segment at the mid range means that for most images, the bulk of the pixels will 
be mapped more or less linearly, thus preserving the shape of the original histogram. In 
addition, the parameters of the sigmoidal tone curve 170 can be set to achieve the desired L * 

15 standard deviation level. Furthermore, the sigmoidal function is smooth, with rather consistent 
curvature across different parameter settings (accelerating before the /? point, decelerating 
after). Therefore, the sigmoidal function is unlikely to generate tone curves 170 that have wild 
bends and peaks. 

[0033] Once the tone curve 170 is generated (step 240), the received linear image data 

20 130 is mapped onto this tone curve 170 by the tone mapping operator 40 to produce output 
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image data 45 corresponding to the levels of the output device 50 (step 250). It should be 
noted that the output device 50 can be a display, printer or other image processing device. It 
should also be noted that additional processing may be performed on the output image data 45 
prior to being sent to the output device 50 (step 260). 
5 [0034] With reference now to FIGURES 3 A-3C of the drawings and the 

corresponding flow charts of FIGURES 4A-4C, alternative embodiments for implementing the 
image statistics processor of FIGURE 1 are illustrated. In a first embodiment, as shown in 
FIGURE 3 A and described in FIGURE 4 A, the linear image data 130 (step 400) is passed to a 
histogram generator 122, which produces a histogram 145 (e.g., a table) describing the 

1 0 distribution of the digital values of the data 130 (step 405). For example, in a color digital 
image, there are a certain number of red, blue and green values (e.g., one of each for each 
pixel in the image). In order to produce a histogram of green values, for example, the 
histogram generator 122 determines a bin size (e.g., range of values, such as 0-10, 1 1-20, 21- 
30, etc.) and counts the number of occurrences of green pixel values in each bin. To create a 

1 5 graphical representation of the histogram, the data in each bin is plotted as a function of the 
bin centers (e.g., 5, 15, 25, etc.). 

[0035] Once the histogram is created, the histogram bin center values 135 are 

transmitted to an L * bin center converter 124 to transform the linear bin center values 135 
into L * bin center values 135a (step 410). The conversion is performed using equation (2) 
20 above, with 7 being the bin center values, and Y„ being the largest possible digital value for the 
pixel values. The L * bin center values 135a and the histogram 145, which includes the count 
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of the number of values in each bin are transmitted to L * statistics calculation logic 128, 
which calculates the image statistics 149 (step 420) that are subsequently used in the 
calculation of the slope parameter a and the shift parameter /? for the sigmoidal tone curve. 
[0036] The various embodiments for calculating the slope parameter a and the shift 

5 parameter fi are discussed below with reference to FIGURES 5 and 6. Briefly, the L * image 
statistics 149 used for calculation of the slope parameter a and the shift parameter /? include 
the current L * standard deviation and either the current mean L * value or the L * percentile 
values of the histogram («th and (100-«)th percentiles). The L * percentile values of the 
histogram represent the end points of the histogram. Therefore, n is chosen to be relatively 
10 small (preferably less than 5). For example, if 5% of all L * pixel values have values less than 
1 8, then the 5 th percentile is 1 8, thus the low end point would be 1 8. The high end point 
would then just be the L * value corresponding to the 95 th percentile (e.g., 95% of all L * 
pixel values have values lower than the high end point). 

[0037] The current L * standard deviation <w en/ can be calculated from the histogram 

1 5 as follows: 

I »t _ 

& current ~ ~ T! («,(#, ~ Lf ) , (EOUatiOIl 4) 

n 

where B,s are histogram bin centers in L * values, n, is the pixel count in the rth bin of the 
histogram, rib is the number of histogram bins, n is the total number of pixels in the image, and 
20 L is the mean pixel L * value, which can also be calculated from the image histogram as 
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follows: 

L = -Y (n. *B,). (Equation 5) 

n ^ 

[0038] In one embodiment, current is calculated from the histogram on the green color 

5 plane only. For most sensors, the green channel is the color plane that approximately captures 
the luminance information. However, it should be noted that some combination of the RGB 
pixel values can be used instead to calculate the image L * standard deviation. The resulting 
tone curve should not vary significantly as long as the choice is a reasonable representation of 
the luminance. 

1 0 [0039] Alternatively, as shown in FIGURE 3B and described in FIGURE 4B, if the 

histogram is not available, the image statistics 149 can be calculated from the image pixel 
values themselves. Therefore, when linear image data 130 arrives at the image statistics 
processor 120 (step 430), the linear image data 130 is converted into L * image values 130a 
by an L * image data converter 125 (step 435), and the L * image values 130a are passed 

15 onto the L * statistics calculation logic 128, where the image statistic 149 (e.g., current L * 
standard deviation and either the current mean L * value or the L * percentiles) are calculated 
(step 440). For example, in this embodiment, the current mean L * value can be calculated by 
taking the average of all of the L * image values. 

[0040] As a further alternative, as shown in FIGURE 3C and described in FIGURE 

20 4C, to reduce computational complexity, since the histogram bins do not change from image 



DALLAS2 760074v3 47429-00014 



-16- 



Patent Application 
PDN# 10010256-1 
Attorney Docket # 47429-00014 

to image (only the counts in each bin change), the histogram bin center values 135 can be 
converted into L * bin center values 135a prior to receiving the linear image data 130. 
Therefore, in this embodiment, once the histogram bin center values 135 are determined (step 
450), the bin center values 135 are converted to L * bin center values 135a by the L * bin 
5 center converter 124 (step 455) and these L * bin center values 135a are stored in a memory 
129 accessible to or within the image statistics processor 120 (the latter being illustrated) (step 
460). As a further alternative, the conversion to L* bin center values 135a can be performed 
externally, and the L * bin center values 135a can be pre-loaded into the memory 129. 
[0041] Thereafter, when the linear image data 130 arrives at the image statistics 

= 10 processor 120 (step 465), the linear image data 130 is sent to the histogram generator 122 
(described above), which counts the number of pixel values in each bin (step 470). Once 
counted, the table 145, which includes the count of the number of pixel values in each bin is 
sent to the L * statistics calculation logic 128, which retrieves the L * bin center values 135a 
from the memory 129 (step 480) to calculate the image statistics 149 (step 485). With this 
15 embodiment, the image statistics 149 (e.g., L * standard deviation and mean L * value) can be 
calculated with only simple addition, subtraction and multiplication. 

[0042] With reference now to FIGURES 5, 6A and 6B, there is illustrated the steps 

for calculating the slope parameter a and the shift parameter /? using the image statistics (e.g., 
L * standard deviation and either mean L * value or L * percentiles) calculated by the image 
20 statistics processor. In calculating the shift parameter and the slope parameter, as discussed 
above, various perceptual preference guidelines can be used to produce an output image that is 
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pleasing to the viewer. The perceptual preference guidelines can be chosen by the user or can 
be selected by the manufacturer of the specific digital image system in which the tone mapping 
algorithm is implemented. As an example, the following discussion is based on three preferred 
perceptual preferences, including the preservation of general histogram shape at the middle 
5 range, a desired L * standard deviation level of around 20 and a desired mean L * level of 
around 50. It should be understood that other perceptual preferences can be used instead of 
these preferred preferences. 

[0043] As illustrated in FIGURE 5, to determine the slope parameter a, the current 

L * standard deviation calculated by the image statistics processor is transmitted to the tone 

10 curve parameter calculation logic (shown in FIGURE 1) (step 500). The desired L * standard 
deviation is pre-selected by the user or the manufacturer and stored within a memory (not 
shown) accessible to or within the tone curve parameter calculation logic (step 510). As 
discussed above, in preferred embodiments, the desired L * standard deviation is chosen to be 
about 20. However, other values can be chosen, depending upon the requirements of the user 

1 5 or the manufacturer. Values greater than 20 produce a more contrasty image, while values 
less than 20 produce a less contrasty image. In addition to the desired L * standard deviation, 
a default slope value is also pre-selected (step 520). 

[0044] With a linear tone curve, the standard deviation can be changed from one value 

(p) to another value (q) by using a tone curve slope of q/p. The sigmoidal tone curve is non- 
20 linear, but the mid-section of the sigmoidal curve is mostly linear. Since for most images, a 
large proportion of image pixels have values close to the mid-range (around the point x-L 
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for one of the embodiments), the slope of the sigmoidal curve can be changed at this point to 
try to achieve the desired L * standard deviation. 

[0045] At the point x = 100/? , the slope of t(x) is 1 when a = 4. Therefore, the 

default slope a de f au i t can be set to 4. However, the actual tone curve used (as discussed above) 
5 is /i(x), which has a slope of 1 when a is less than 4. In practice, any value between 3 and 4 
can be used as the default slope Odefauit, depending on how contrasty one would like the image 
to look. Once the default slope has been selected, the slope parameter a can be calculated 
based on the current L * standard deviation, the desired L * standard deviation and the 
default slope (step 530). To get the slope parameter a for an image with an I* standard 
10 deviation of <w ren/ , the default a value is scaled according to the current L * standard 
deviation and the desired L * standard deviation <5 destr ed as follows: 

« = "curnt x 1*=*- • (Equation 6) 

[0046] As mentioned above, the shift parameter ft shifts the histogram to affect the 

1 5 general brightness of the output image. There are different ways to determine the shift 
parameter, depending on what perceptual factors are more important in the particular 
application, and what computational complexity is allowed. FIGURES 6A and 6B illustrate 
two such alternative embodiments for calculating the shift parameter /?. 
[0047] In a first embodiment, as shown in FIGURE 6 A, the shift parameter /? is 

20 calculated based on the mean L * value. In this embodiment, the current mean L * value 
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calculated by the image statistics processor is sent to the tone curve parameter calculation 
logic (shown in FIGURE 1) (step 600). The desired mean L * value is pre-selected by the 
user or the manufacturer and stored within a memory (not shown) accessible to or within the 
tone curve parameter calculation logic (step 610). Under some circumstances, however, this 
5 pre-selected desired mean L * value may be adjusted based on additional information about 
the image being processed. For example, if the content of the image is known to the image 
processor to be predominantly black objects, then the desired L * value can be adjusted to be 
lower than 50, to avoid over-brightening the image. Or, if the image data is known to have a 
high level of noise, the desired L * value can be biased to be closer to the original image mean 

10 L * value, to avoid amplification of the noise through tone mapping. 

[0048] For example, the desired mean L * value of the whole image can be pre- 

selected to be around 50 (medium brightness). Alternatively, the desired mean L * value can 
be selected to be 50 in a certain area of the image. As an example, for images containing a 
human face, the L * mean value at the face area can be set to 50. Using such criterion to 

15 determine the shift parameter of a sigmoidal tone curve generally results in a good quality 
image. However, when face recognition is not feasible, the mean L * value of the center part 
of an image, or some other area of interest in the image, can be brought to a level of around 
50. This latter method of determining the shift parameter is more practical for a low cost 
imaging system. It should be understood that any other desired mean L * value can be used 

20 instead of 50. 

[0049] The shift parameter can be calculated based on the current mean L * value and 
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the desired mean L * value (step 620). For example, if L is the mean L * value of the area 
of interest in an image (e.g., a face area), and Laired is the desired mean L * value, the shift 
parameter is equal to: 

P = (L~ L destred )/100 + 0.5 . (Equation 7) 

5 

When using an Ld es!r ed value of 50, ft is simply Z./100 . 

[0050] In a second embodiment, shown in FIGURE 6B, the shift parameter ft is 

calculated based on the L * percentile values. Various studies {see, e.g., Tanaka et al., 1997) 
have suggested that observers preferred renderings that have as large an output gamut as 

10 possible. Therefore, to accommodate these preferences, the shift parameter can be determined 
by shifting the histogram to make it occupy as much of the output device dynamic range as 
possible, while bringing as little disturbance to the shape of the histogram as possible to avoid 
saturation of a lot of points at either end of the distribution. This is done by using a centering 
function in which the tone curve is centered at midway between the end points of the 

15 histogram. This is a conservative choice, typically resulting in only small changes to the mean 
luminance of the image if the original RGB histogram is somewhat "normal" (values are not 
concentrated on a small section of the full range). 

[0051] Therefore, when the tone curve parameter calculation logic receives theZ, * 

percentiles (e.g., nth. and (100 - «)th percentiles, with n being a convenient small value) (step 
20 650), the shift parameter ft can be calculated as the mid-point between these two end points 
(step 660). For example, if the nth percentile of pixel values in the histogram is Xi, and the 
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(100-w)th percentile of pixel values is x 2 , then the shift parameter is determined by the 
following equation: 

j3 = h_±^ (Equation 8) 

5 The combined histogram of RGB values is preferably used to determine the percentiles, so that 
no color plane is ignored when calculating the shift of the tone curve. This is especially 
important for tone mapping performed on images with a strong color cast, to avoid color 

shifts after tone mapping. 

[0052] Both methods of determining J5 discussed above have been implemented and 

10 tested on images. The first method (fi calculated from mean L * value) is simpler to 

implement, and works well on most indoor and outdoor images. When the mean L * value of 
the whole image is used as it tends to over-brighten night-time or dark images, i.e., images 
that are meant to have a low mean L * value, such as candlelight scenes. The second method 
(using the average of the top and bottom percentile of the histogram as /?) works better for 
15 dark images. It tends to change the image brightness less than the first method, so it is a more 
conservative method. 

[0053] For some applications, it may be desirable to limit the extent that the sigmoidal 

curve is shifted, i.e., the extent the shift parameter ft can deviate from 0.5. For example, for 
severely under-exposed images, excessive shifting up of the tone curve (fi value close to 0) can 
20 result in a low-quality image due to amplification of sensor noise. Likewise, excessive shifting 
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down of the tone curve (ft value close to 1) for highly over-exposed images can produce color 
shifts in nearly saturated areas. To minimize these problems, the value of the shift parameter 
can be limited depending on the mean pixel values of the original image. If the original image 
mean is too high or too low, depending on the specific application of the tone mapping 
5 algorithm, then the shift parameter ft is adjusted to deviate less from the midpoint 0.5 in order 
to reduce noise amplification. In one embodiment, a series of ft value upper limits 7} are pre- 
selected, each corresponding to one mean level M } of the original image pixel values. When 
the original image mean pixel value is less thanM,, then the ft value is limited to be at least T } , 
i.e., if the value calculated from Equation 7 or Equation 8 is less than 7), then ft is set to be 
10 equal to T } . Similarly, a series of lower limits T k can also be pre-selected, each corresponding 
to one mean pixel level M k . When the original image mean pixel value is larger than A4, then ft 
is limited to be no greater than T k . By limiting the shift parameter in this way, artifacts 
resulting from tone over-compensation can be minimized. 

[0054] With reference now to FIGURES 7A and 7B, there are illustrated alternative 

15 embodiments for implementing the tone curve generator 160 shown in FIGURE 1 of the 
drawings. In a first embodiment, as shown in FIGURE 7A and described in the steps of 
FIGURE 8A, when the tone curve parameters 150 (e.g., slope parameter a and the shift 
parameter ft) have been calculated by the tone curve parameter calculation logic 140 (shown in 
FIGURE 1), these parameters are sent to the tone curve generator 160 (step 800). Thereafter, 
20 the sigmoidal tone curve 170 is generated by sigmoidal function logic 162 within the tone 
curve generator 160 using the received tone curve parameters 150 and equations (1) and (3) 
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listed above (steps 805 and 810). Once generated, the resulting tone curve 170 resembles the 
tone curve shown in FIGURE 9. 

[0055] The tone curve 170 shown in FIGURE 9 is capable of mapping L * image 

input values onto L * image output values for the appropriate output device. However, since 
5 L * is not a commonly used scale in output devices, the output digital image data is converted 
to linear form before conversion to the scale required by specific output devices. In preferred 
embodiments, the Y-axis (L* image output values) is converted by a Y-axis converter 164 
from an L * scale to a linear scale (step 815), which is approximately a power of three 
transformation, as discussed above, to produce tone curve 170a. 

10 [0056] In addition, since most display devices, such as a CRT display, have an 

intensity to voltage response curve that is roughly a power function (e.g., if the pixel value is 
set at x 5 the displayed luminance is proportional to x r , with y being the display's gamma 
value), the linear output values need to be gamma corrected in order to be displayed properly 
on such output devices. In most tone mapping methods, gamma correction is performed after 

1 5 mapping. In the sigmoidal tone mapping method described herein, gamma correction can 

either be performed separately after tone mapping, or can be combined with the tone mapping 
step. In one embodiment, the gamma correction is performed together with tone mapping by 
applying gamma correction logic 166 to the Y-axis to produce tone curve 170b (step 820). 
The combined effect of linear transformation and gamma correction of the Y-axis (output) is a 

20 power of 3 ly , where y is the intended display gamma value, such as 2.4, if the output device 
is an sRGB (standard color space) display. 
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[0057] Typically, the tone mapping is applied to linear RGB input pixel values (e.g., 

linear image data 130 is supplied by the linear transformation device 30 to the tone mapping 
operator 40 in FIGURE 1). The sigmoidal tone curve is generated assuming an L* scale on 
the X-axis, i.e., X-axis values uniformly sampled on the L * scale. Therefore, in order to 
5 properly map linear input values onto the tone curve, the tone curve's X-axis sampling has to 
be converted to be uniform on a linear scale for easy table lookup operation when applying the 
tone curve to images. This is done by an X-axis converter 168 to produce tone curve 170c 
(step 825), using a simple linear interpolation process, which requires only arithmetic 
operations, as described above in connection with the Y-axis converter. This final tone curve 
10 170c is used by the tone mapping operator (shown in FIGURE 1) to map the linear image data 
onto the luminance of the output device. 

[0058] In a second embodiment, as shown in FIGURE 7B and described in the steps of 

FIGURE 8B, since one of the most computationally costly parts of the tone mapping 
algorithm involves the exponential function calculation in the sigmoidal function, to reduce 

15 computational complexity, an approximation of the sigmoidal tone curve calculation can be 
implemented with only arithmetic operations. The exponential calculation in the sigmoidal 
function can be avoided by pre-calculating a pair of fixed tone curves with two extreme slopes, 
and then interpolating to obtain tone curves with intermediate slopes. As discussed above, the 
shift parameter translates the tone curve on the horizontal (X) axis. Therefore, by pre- 

20 calculating the tone curves for a wider range on the X (input pixel value) axis, appropriate 
tone curves can be selected by "cutting out" different sections of the pre-calculated tone 
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curves for different shift parameters. 

[0056] To pre-calculate the two tone curves, the user or manufacturer must pre-select 

two slope parameters, owx and a^n (step 850). The slope parameter a for most images falls in 
a range from 2.5 to 6.5. Therefore, in preferred embodiments, ou is set to be 2.5 and a miX is 
5 set to be 6.5. Once the maximum and minimum slope parameters 155 are set, sigmoidal 
function logic 162 can pre-calculate the two tone curves (step 855), one with a slope of 2.5 
(curve Pi), which gives an almost linear tone curve (on the L * scale), and one with a steep 
slope of 6.5 (curve P 2 ), which gives a lot of contrast enhancement to the image. The two pre- 
generated tone curves are shown in FIGURE 10. It should be noted that the two curves are 
10 generated for a shift value y? of 0. (The X-axis range for different /? values are later determined 
by translating the X origin on the X-axis, as discussed hereinbeiow.) 
[0057] For slope values between 2.5 and 6.5, the tone curve is generated by linear 

interpolation between Pi and P2. Let P a (x) be the tone curve with an a value of a, with a c 
[2.5, 6.5], then: 

1 5 P a (x) = /» (x) + (P 2 (x) - /> (x)) * ° ~ \\ . (Equation 9) 

o.-> — Z.j 

[0058] This of course is an approximation, but for the limited a value range between 

2.5 and 6.5, the approximation is reasonably accurate. In FIGURE 1 1 A, the tone curve 
output L * values are plotted as a function of a for 1 1 different input pixel levels (represented 
20 by the 1 1 lines). As can be seen, the output L * values are close to linearly related to a. 
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[0059] As discussed above, typically, the tone mapping output is gamma-corrected 

linear RGB values ready for display on a screen. Therefore, as described above in connection 
with FIGURES 7A and 8A, the Y-axis of Pi and P 2 1 58 may be converted to gamma- 
corrected linear RGB values (steps 860 and 865) by the Y-axis converter 164 and gamma 
5 correction logic 166 of the tone curve generator 160. When the tone curve output P a (x) is 
gamma corrected linear RGB values, the relationship with a is still quite linear, as shown in 
FIGURE 1 IB. Therefore, by pre-calculating Pi and P 2 158 as gamma corrected pixel values, 
the interpolated tone values will be gamma corrected as well. This means the tone curve 
calculation only requires the arithmetic operations used in interpolation (equation 9 above). 
10 Once the gamma corrected tone curves Pi and P 2 158 have been pre-generated, they are 
stored in a memory 169 (step 870) accessible to or within the tone curve generator for later 
use in calculating the tone curve for a particular set of input values. 

[0060] When the tone curve generator 160 receives the current slope parameter a cwrsnt 

150a and the current shift parameter /3 cvmn t 150b (step 875) from the tone curve parameter 

15 calculation logic (shown in FIGURE 1), calculation logic 163 within the tone curve generator 
160 determines P a (x) 159 using the gamma corrected tone curves Pi and P 2 158 and the 
current slope parameter 150a and equation 9 above (step 880). Thereafter, to select a section 
of P a (x) , the current shift parameter 150b and P a (x) 159 are transmitted to shifting logic 165 
within the tone curve generator 160 (step 885). Since the shift parameter 150b simply 

20 translates the tone curve on the X-axis, tone curves with different shift parameters can be 
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obtained by translation on the X-axis of P a (x) . 

[0061] For example, assuming P a (x) is generated with a shift parameter of 0 and in 

the range [-100, 100], to get a tone curve with a shift parameter of fi at the X-axis range of [0, 
100], the tone curve is translated on the X-axis so that an X value of - fi * 100 now becomes 
5 0, and an X value of (1 - J3) * 100 becomes 100. The tone curve with shift parameter /? is 
obtained by "cutting out" the pre-generated tone curve PJx) from the point X - - fl * 100 to 
the point X = (1 - /?) * 100 . In implementation, this operation involves simply moving the file 
pointer to different starting points in a pre-generated table, thus requiring almost no 
computation. 

10 [0062 j After the tone curve is "cut out" from P a (x), the two ends of the tone curve 

need to be scaled to the full Y range. This again only requires arithmetic operations on each 
entry of the tone mapping table that was previously cut out from the pre-generated table. 
Thereafter, as discussed above in connection with FIGURE 7A and 8A, the X-axis converter 
168 converts the X-axis from an L * scale to a linear scale (step 890) to produce the final tone 

15 curve 170. It should be noted that the translation of the tone curve can happen before or after 
the interpolation step required to get the correct slope by translating tone curves Pi and P 2 
158. For the least amount of computation, the translation is performed first, so that the 
interpolation only needs to be done on the "cut out" section of the pre-generated tone curves 
P, and 7*2 158. 
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[0063] As will be recognized by those skilled in the art, the innovative concepts 

described in the present application can be modified and varied over a wide range of 
applications. Accordingly, the scope of patented subject matter should not be limited to any of 
the specific exemplary teachings discussed, but is instead defined by the following claims. 
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